Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat] Support WKB geometry column in CSV #1

Merged
merged 9 commits into from
Sep 15, 2023
Merged

Conversation

lixun910
Copy link
Member

@lixun910 lixun910 commented Sep 7, 2023

In Kepler.gl, geometries (Polygons, Points, LindStrings etc) can be embedded into CSV as a GeoJSON or WKT formatted string. (see here). This PR is to support WKB geometry column in CSV file. For example, the following CSV row contains a WKT geometry:

id,geom
2,POINT(1 2)

the following CSV row contains a WKB geometry in hex format:

id, geom
1,0101000000000000000000F03F0000000000000040
  1. The WKB format is more efficient than the WKT for storing and processing geometry objects. e.g. the output of the geometry column in PostGIS is returned as WKB in hex format by default unless using e.g. ST_AsWKT
  2. The WKB format keeps the precisions of the raw geometry data
  3. The size of WKB in hex format is 2 times the size of WKB, however, it is usually smaller than WKT with complex geometries (linestrings, polygons etc. -- don't use WKB hex to store simple points : )
  • guerry.geojson is 2 MB
  • guerry_wkt.csv is 1.7 MB
  • guerry_wkb.csv is 1.4 MB
  • guerry.arrow is 705 KB

There is another wip PR to support Arrow/GeoArrow in Kepler.gl: #2

@lixun910 lixun910 force-pushed the xli-misc-fix-ts-config branch 4 times, most recently from c6be6e8 to 9d2bcfa Compare September 15, 2023 04:50
@coveralls
Copy link

coveralls commented Sep 15, 2023

Pull Request Test Coverage Report for Build 6201115311

  • 11 of 11 (100.0%) changed or added relevant lines in 2 files are covered.
  • 3 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.006%) to 78.803%

Files with Coverage Reduction New Missed Lines %
src/layers/src/geojson-layer/geojson-utils.ts 3 84.0%
Totals Coverage Status
Change from base Build 6166025427: 0.006%
Covered Lines: 10950
Relevant Lines: 12848

💛 - Coveralls

Signed-off-by: Xun Li <[email protected]>
Signed-off-by: Xun Li <[email protected]>
Signed-off-by: Xun Li <[email protected]>
Signed-off-by: Xun Li <[email protected]>
Signed-off-by: Xun Li <[email protected]>
Signed-off-by: Xun Li <[email protected]>
Signed-off-by: Xun Li <[email protected]>
@lixun910 lixun910 merged commit 7ce6a85 into master Sep 15, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants